Repository navigation
Fix sign of poisson noise in preprocess.poisson_noise_via_data_eps_from - #290
Merged
Merged
Conversation
The function returned data_eps - noisy_eps, which made data_eps_with_poisson_noise_added produce 2*data - noisy_data (noise reflected around the mean — same magnitude, mirrored skew). Poisson is positively skewed; the buggy version had negative skew, biasing inference on simulated low-count imaging. Now returns noisy_eps - data_eps so that data + noise = noisy_data. Also wraps the result back into the input Array2D type via with_new_array(), preventing the numpy/Array2D dispatch from returning a raw ndarray. Test fixtures regenerated under the same seeds. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
autoarray.preprocess.poisson_noise_via_data_eps_fromreturned the wrong sign of the Poisson noise term. It computeddata_eps − noisy_epsinstead ofnoisy_eps − data_eps, sodata_eps_with_poisson_noise_added(...)andSimulatorImaging.via_image_from(..., add_poisson_noise_to_data=True)produced2·data − noisy_datarather thannoisy_data. Same mean and variance as a real Poisson sample but with mirrored skew (negative instead of positive), which biases inference on simulated low-count imaging.Surfaced by
/health_checkon 2026-04-27.API Changes
Behaviour-only change to two existing functions (signatures unchanged):
preprocess.poisson_noise_via_data_eps_fromnow returns the true Poisson noise term (noisy − data) instead of its negation.preprocess.data_eps_with_poisson_noise_addedandSimulatorImaging.via_image_from(..., add_poisson_noise_to_data=True)therefore produce data with the correct Poisson distribution and skew — same seed, different numerical values.Implementation also routes the result through
data_eps.with_new_array(...)so the return type is anArray2D(not a rawnp.ndarray), which the previous in-progress edit had inadvertently changed.Any code asserting against specific seeded outputs of these functions will need expected values regenerated. Anything using the result as a noisy realisation (modelling, inference) will now see correctly-skewed samples — typically more accurate, not less.
See full details below.
Test Plan
pytest test_autoarray/dataset/test_preprocess.py test_autoarray/dataset/imaging/test_simulator.py— 34/34 greenpytest test_autoarray/— 748/748 green/smoke_test)Full API Changes (for automation & release notes)
Changed Behaviour
autoarray.preprocess.poisson_noise_via_data_eps_from(data_eps, exposure_time_map, seed=-1)— sign of returned noise corrected. Wasdata_eps − noisy_eps, nownoisy_eps − data_eps. Result is now correctly typedArray2D(matches input).autoarray.preprocess.data_eps_with_poisson_noise_added(...)— downstream consequence. Result is now a true Poisson realisation ofdata_epsrather than its mirror image around the mean.autoarray.SimulatorImaging.via_image_from(...)— whenadd_poisson_noise_to_data=True, simulated data and (wheninclude_poisson_noise_in_noise_map=True) the noise map both shift to correct Poisson statistics.Migration
🤖 Generated with Claude Code